FOVEATED DISPLAY SYSTEM 

Field of the Invention 

[0001] This invention generally relates to immersive dome 
theaters, including large-format film theaters and video-based 
digital dome theaters which are generally spherical in shape, and 
which can be hemispheric, hypohemispheric, or hyperhemispheric in 
extent • 

Background of the Invention 

[0002] A typical installation for a modern dome theater is 
shown in Figure 1. Most modern dome theaters 1 utilize a dome 2 
which is tilted forward and which has unidirectional seating 3 
mounted on a stadium-type seating deck 4. An omni-type, large 
format film projector 5 is then situated within the theater 
space 6 which is defined by the dome 2. Examples of such dome 
theaters would include the IMAX™ Dome system, available from 
Imax Corporation, and the Cinedome'^" format available from Iwerks. 
[0003] The projector 5 is generally positioned behind the 
center 7 of the dome 2 or, as an alternative, at the center 7 of 
the dome 2, and typically uses a "fisheye" type lens (not shown) 
that has a very large f ield-of-view (e.g., 150 to 180 degrees). 
The projector 5 is accessed from a sub-floor 8 (i.e., from under 
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the seating deck 4) . A so-called "dog house" 9 is provided to 
contain the projector 5, which typically protrudes into the 
theater space 6. This leads to the disadvantage that valuable 
seating space is taken up in a prime viewing location, that being 
the center of the theater 1. 

[0004] Another major problem with modern dome theaters is 
caused by scattered light. Since the projection screen (the dome 
2) is curved, the image projected onto one portion of the screen 
can scatter light onto remaining portions of the screen. This 
leads to the disadvantage that the contrast of the resulting 
image can be reduced. 

[0005] Another problem associated with modern dome theaters 
is the difficulty of obtaining sufficient image resolution, 
primarily due to the extremely wide f ield-of-view of the 
projection system. Eye-limited resolution over a hemispheric 
f ield-of-view requires a large number of pixels. This number 
is given by the following equation: 

Number = (27t steradians) ( 57 . 3°/steradian) ^ ( 60 arc min/deg)^ = 300 million 
of pixels (0.5 arc min/pixel)^ pixels 

No medium is presently known which is capable of achieving this 
resolution. For example, large-format films (15 perf/70mm) are 
capable of an effective pixel resolution on the order of 10 to 12 
million. Video projectors, however, are currently only capable 
of an effective pixel resolution on the order of 3 million. 
[0006] The limited resolution of video projectors has 
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prevented the use of "single-lens" projection schemes. Instead, 
multiple, edge-blended projectors have been required to achieve a 
large, seamless, high-resolution image. Such systems, however, 
have the disadvantages of being expensive and difficult to 
maintain. It is also difficult to format content for such 
systems because the images must be split into separate frames, 
for projection by individual projectors, which must carefully 
reconstruct the image on the dome screen using edge-blending 
techniques . 

[0007] Another problem associated with modern dome projection 
systems is that the brightness of the image tends to be somewhat 
limited. While flat-screen theaters easily fulfill the current 
SMPTE cinematic standard for brightness of 12 foot-Lamberts, dome 
theaters usually provide a brightness on the order of 3 to 4 
foot-Lamberts, or less, due to their large image area and the 
limited brightness of the projectors. 

Summary of the Invention 

[0008] The present invention provides an improved theater 
geometry which is capable of providing improved image resolution 
and improved image contrast over prior systems. This is achieved 
with a unique projection geometry and image re-mapping technique. 

[0009] The improvements of the present invention exploit the 
known phenomena that the human visual system is "foveated", i.e., 
that the central f ield-of-view of the human eye has a higher 
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resolution than its peripheral f ield-of-view (see, J.M. Rolfe, 
et al., Flight Simulation, Cambridge Press, New York, 1986), and 
that most people exercise limited head motion when viewing films, 
even when given a fully immersive f ield-of-view (see, R. Pausch, 
et al., "Disney's Aladdin: First Steps Toward Storytelling in 
Virtual Reality", ACM SIGGRAPH 96 Conference Proc. , August 1996). 
The combined effect of this is that image resolution is most 
important in the central region of the dome screen, and less 
important at the sides and the rear of the dome screen. 
[0010] In accordance with the present invention, the 
projected image is provided with a continuously variable 
image resolution and brightness over the surface of the dome, 
in this way concentrating the resolution and the brightness 
of the image within the central f ield-of-view of viewers that 
are unidirectionally seated in the theater, and sacrificing 
resolution and brightness toward the outside edges of the 
viewers' f ield-of-view. The result is a more efficient use 
of available projector resolution and brightness, an increase 
in the number of quality seats available in the theater, and 
an enhanced image contrast due to reductions in the light which 
is scattering from image elements to the rear of the dome. 

Brief Description of the Drawings 

[0011] Figure 1 is a schematic view of a typical omni-format 
film theater. 
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[0012] Figure 2 is a schematic view of a theater having a 

projector which is located for producing a foveated display in 
accordance with the present invention. 

[0013] Figure 3 is a schematic view showing the reconstruction 
and projection of an image which has been foveated in accordance 
with the present invention, for a spherical f ield-of -view. 
[0014] Figure 4 is a schematic view showing the eyepoint 
projection for generating an image which has been foveated in 
accordance with the present invention. 

[0015] Figure 5 is an azimuthal equidistant map showing a 
hemispheric grid image which has been foveated in accordance 
with the present invention . 

Detailed Description of tlie Invention 

[0016] Figure 2 shows an installation for a dome theater 10 
having a display which has been "foveated", in accordance with 
the present invention. As with the theater 1 shown in Figure 1, 
the theater 10 utilizes a dome-shaped screen 11 {hereafter also 
referred to as the "dome") which is tilted forward, and has 
unidirectional seating 12 which is mounted on a stadium-type 
seating deck 13. An omni-type, large format projector 14 (either 
a film or a video projector) is situated within the theater space 
15 which is defined by the dome 11. Such theaters can again 
include theater systems such as the previously mentioned IMAX™ 
Dome system, available from Imax Corporation, and the Cinedome''" 

-5- 



format available from Iwerks, as well as other theater systems 
that may be in existence or that may be developed in the future. 
[0017] In accordance with the present invention, the projector 
14 is placed substantially in front of the center 16 of the dome 
11. A "fisheye" type lens 17 (having a very large f ield-of-view 
on the order of 150 to 180 degrees) is again coupled with the 
projector 14, and the source image is specially mapped to correct 
for distortion created by parallax in the projector, as follows. 
[0018] Pixels in the front of the dome screen 11 (at 18) are 
made smaller, resulting in a higher resolution in the direction 
that the viewers are facing (shown by the arrow 19) . Conversely, 
the pixels in the rear of the dome screen 11 (at 20) are made 
larger. For a given projector resolution, the projected 
resolution is effectively sacrificed in the rear of the theater 
(at 20, behind most of the viewers) to boost the resolution in 
the front of the theater (at 18). This results in a higher 
perceived resolution, since the pixels in the rear 20 of the 
theater 10 will lie in the outer edges of the viewers' peripheral 
vision, where visual acuity is generally the lowest, and do not 
substantially contribute to the quality of the resulting image. 
As a result, the pixels in the rear 20 of the theater 10 are not 
perceptibly missed. Pixels are instead concentrated in the front 
18 of the theater 10, where visual acuity is generally the 
highest and the viewers' attention is most focused. 
[0019] The brightness of the image at the front 18 of the dome 
screen 11 is increased, while the brightness of the image at the 
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rear 20 of the dome screen 11 is decreased. This decrease in the 
brightness of the image, toward the rear 20, improves contrast in 
the central, high-resolution regions (the center 16 and the front 
18) since less light is scattered from scenes in the rear 20 
onto the front 18 of the dome screen 11, Again, this drop in 
brightness occurs in regions of the dome screen 11 that lie on 
the outer fringes of a viewer's peripheral vision, which does not 
substantially detract from the projected image. 

[0020] The projector 14 is preferably located in the front 21 
of the theater 10, possibly in front of the first row of seats 
22. This permits a greater number of seats to be placed in the 
theater 10, particularly in the center 23 of the seating deck 13 
(i.e., at the center of the theater), seats which had previously 
been taken up by the projector-receiving "dog-house" 9. This, in 
turn, restores the highest quality seats to the theater 10. 
[0021] In the case of a video projector, the increased 
resolution and brightness in the prime image area can be 
sufficient to achieve cinematic quality, allowing a single 
projector to replace the less reliable, multi-projector, 
edge-blended video display systems currently in use. 
[0022] Because the foveated effect is achieved using a single 
projector 14, with a fisheye lens 17, the drop in brightness and 
resolution from the front 18 of the dome screen 11 to the rear 20 
of the dome screen 11 is graceful and slowly varying. Provided 
the displacement of the projector 14 from the center 16 of the 
dome 11 falls within a desired range of values, it will be 
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difficult for the drop in brightness and resolution to be 
noticed by the untrained eye. 

[0023] Figure 3 generally illustrates the operating principal 
of a display which has been foveated in accordance with the 
present invention. Since the front 18' of the dome 11' is closer 
to the projection lens (schematically shown at 24), the pixels 
at the front 18' of the dome 11' are smallest and brightest. 
Conversely, the pixels at the rear 20' of the dome 11' are 
the largest, because the length of the projection path 25 for 
the pixels at the rear 20' of the theater is the longest. The 
greater the displacement (d^,) of the projection (fisheye) lens 
24 from the center 16' of the dome 11', the greater the foveated 
effect that will result. 

[0024] A normalized projector displacement ratio (R) is given 
by the ratio d^/r. Consequently, the size of the pixels in the 
front 18' of the theater will be scaled down by a factor Sf, 
where Sf = (r-dr)/r, with respect to conventional projection 
from the center 16' of the dome 11' (dr=0) , while the size of 
the pixels at the rear 20' of the theater will be scaled up by 
a factor S^, where S^ = (r+dr)/r, relative to projection from the 
center 16' of the dome 11'. A measure of the degree of foveation 
is provided by the "f ront-to-back" resolution ratio (F) which is 
given by the ratio F = S^/Sf. Foveation is the ratio of the 
resolution (and brightness) of the image in the front 18' of 
the theater versus the resolution (and brightness) of the image 
in the rear 20' of the theater. 
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[0025] It can also be shown that the critical f ield-of-view 
(measured at the equator of the dome, shown at 26 in Figure 4, 
with respect to an eyepoint 27 at the center 16' of the dome 
11') for which the pixel resolution equals that of conventional 
(dome-centered) projection, 0^, is given by the equation 
2cos"MR/2) . Within this f ield-of -view, the pixel resolution 
(and image brightness) will exceed that of conventional, 
dome-centered projection. Outside of this f ield-of-view, the 
pixel resolution (and image brightness) will drop below that of 
conventional, dome-centered projection. The value of 0c is an 
important parameter for providing a foveated display since it 
dictates the area on the dome for which an elevated brightness 
and resolution are achieved. 

[0026] The following table (Table 1) provides an example 

of a list of foveated display parameters versus a projector 
displacement ratio, R. 

Table 1 



Projector 


Critical 


Normalized 


Normalized 


Foveation 


Displacement 


Field of 


Front Pixel 


Back Pixel 


F = Sb/Sf 


Ratio, R 


View, 


Size, Sf 


Size, Sb 




0 


Full Dome 


1.0 


1.0 


1.0 


0.1 


174° 


0.9 


1.1 


1.22 


0.2 


169° 


0.8 


1.2 


1.50 


0.3 


163° 


0.7 


1.3 


1.86 


0.4 


157° 


0.6 


1.4 


2.33 


0.5 


151° 


0.5 


1.5 


3.00 


0.6 


145° 


0.4 


1.6 


4.00 


0.7 


139° 


0.3 


1.7 


5.67 


0.8 


133° 


0.2 


1.8 


9.00 


0.9 


127° 


0.1 


1.9 


19.00 


1.0 


120° 


0 


2.0 


Infinity 
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The values of 9c, S^, and F are listed in Table 1 as a 
function of the projector displacement ratio (R) . Note that, 
even as R approaches 1.0 (i.e., the projector lens 17 approaches 
the front surface 18' of the dome 11*)/ Qc does not drop below 
120°. The corresponding foveation (F) increases exponentially 
with the projector displacement (R) . Lower values of F (e.g., 
for F < 1.5) will not produce a noticeable foveation effect, 
while higher values (e.g., F > 6) will likely be objectionable, 
depending on the seating arrangement, image content, and other 
factors. Since brightness variations due to foveation will 
likely be more noticeable than resolution variations, a 
brightness compensation mask can be applied to the image, 
either optically or electronically, to create a more even 
image brightness over the dome screen by gradually attenuating 
brightness in the front of the image area. 

[0027] To successfully achieve the above-described, foveated 
effect, special image re-mapping is required to result in a 
sphere having a foveated view. Referring to Figure 4, the 
image re-mapping which is required to achieve a sphere 28 
having a foveated view (based on an original, un-foveated view 
sphere 29) is based on a three-dimensional eyepoint projection. 
The foveated image 30 which results is shown in Figure 5, in 
equidistant polar mapping with a hemispheric grid. It is to be 
noted that the larger image area (i.e., having a higher pixel 
density) is found at the front 18' of the dome 11'. In its 
preferred embodiment, digital image processing is used as the 
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mapping algorithm. With the source image in a digital format, 
the algorithm is performed digitally, either in real time for 
immediate projection, or as an off-line process. In an off-line 
process, the resulting frames can be output to a digital video 
storage medium for video-based projection, or scanned onto film 
in the case of a film-based projection theater. 

[0028] Eyepoint mapping procedures which are otherwise known to 
persons of ordinary skill in the fields of geometric mapping and 
software development can be used to implement the foregoing image 
re-mapping procedures, based on the following description. An 
original spherical image is mapped onto a virtual dome (the 
dome 29 shown in Fig. 4) . The pixels on this view sphere (the 
circles 31 shown on the surface of the dome 29 in Fig. 4) are 
geometrically projected onto the foveated view sphere (the dome 
28 shown in Fig. 4), which is displaced by the ratio (R) • The 
foveated view sphere is then prepared appropriately for fisheye 
projection. Typically, this preparation is a 2-dimensional, 
equidistant azimuthal or polar mapping. In practice, the eyepoint 
mapping can be performed as a 2-dimensional image warping 
operation (note, for example, the mapping shown in Fig. 5) . 
[0029] It will be understood that various changes in the 
details, materials and arrangement of parts which have been 
herein described and illustrated in order to explain the nature 
of this invention may be made by those skilled in the art within 
the principle and scope of the invention as expressed in the 
following claims. 



